Field of the Invention
The present invention relates to an image pickup apparatus, an image processing method, and a recording medium for refocus processing.
Description of the Related Art
Each of Ren Ng et al., “Light Field Photography with a Hand-held Plenoptic Camera,” 2005 Computer Science Technical Report CTSR, Todor Georgiev et al., “Superresolution with Plenoptic 2.0 Camera,” 2009 Optical Society of America, Aaron Isaksen et al., “Dynamically Reparameterized Light Fields,” ACM SIGGRAPH, pp. 297-306 (2000) discloses a differently structured image pickup apparatus configured to simultaneously obtain both a two-dimensional intensity distribution of light and angular information of a ray in an object space using the “light field photography.” Herein, the two-dimensional intensity distribution of the light and the angular information of the ray will be collectively referred to as a light field, and three-dimensional information can be acquired by obtaining the light field. The above image pickup apparatuses can provide a change of a focused position of an image, which is referred to as refocus, a change of a capturing viewpoint, and an adjustment of a depth of field, etc. using the light field and image processing to the captured image.
Japanese Patent Laid-Open No. 2011-109310 discloses an applied example of a refocus function as assistance of autofocus. Japanese Patent Laid-Open No. 59-146029 discloses a method for capturing an image by shifting a focused position with a depth of field determined by a diaphragm so as to include a plurality of objects in the depth of field. Japanese Patent No. 4766133 discloses a method for enabling many objects to be refocused by calculating an object distribution on an image plane.
The above refocus function enables a focused position in a captured image to be changed, but a refocusable range is limited (depending upon the specification of the image pickup apparatus) and a focused position cannot be changed to a fully arbitrary position. In addition, a distance in the optical axis direction of a front (camera side) refocusable range of a position focused by a focus unit is different from that of a back (background side) refocusable range of the position focused by the focus unit. Hence, a user may not change the focused position in the captured image as desired. Japanese Patent Laid-Open No. 2011-109310 discloses a method for shifting the focus position using the focus unit when the object is located outside the refocusable range, but is silent about a method for effectively utilize the refocusable range.
In order to capture an image focused on a plurality of objects, Japanese Patent Laid-Open No. 59-146029 shifts the initial focused position to another focused position before capturing an image, and then captures the image. This method can provide the image focused on the plurality of objects (with a deeper depth of field than that with a single object), but cannot provide an image in which a user desired position is focused and other parts are defocused. This reference is also silent about the method for effectively utilizing a refocusable range.
Japanese Patent No. 4766133 calculates an image plane shift amount for each focus detecting position on the image plane, specifies areas of the objects by considering a distribution of the image plane shift amount, and controls the focus unit so that all objects are included in the refocusable range. However, as the number of focus detecting positions increase, the calculation cost used to calculate the image plane distribution increases. Moreover, the object calculated from the image plane shift distribution may not necessarily be an object intended by the user. This reference is also silent about the method for effectively utilizing a refocusable range.